Web Survey Bibliography
This paper investigates potential improvements that can be made to online survey experiments using pairing and blocking techniques, as opposed to simple random assignment. While random assignment studies hold the assumption that control and treatment groups have the same characteristics, statistical variability causes this to not always be the case in practice. With some frequency, control and treatment groups end up being significantly different on one or more variables of interest, simply by chance. As a solution, matched pair and block experiments create control and treatment groups that are more similar than they would be by purely random allocation. In matched pair experiments, respondents with similar characteristics are “paired” and assigned to separate treatments, thereby ensuring that the control and experimental groups are statistically identical. This design leads to increased efficiency in treatment effect estimates (Randomized blocking is a more general case of this type of design).In a recent YouGov-Polimetrix survey, we implemented survey software that assigns respondent pairs and blocks to separate treatment groups automatically, while avoiding the predictable assignment problem. Using data from this survey, we compare the efficiency between randomized pair, randomized block, and simple random assignment experiments. Additionally, we examine if any types of unexpected bias are introduced by using randomized pair and block designs instead of traditional methods.
Homepage(abstract)